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My  paper  will  first  record  what  1  take  to  be  the  main  limita¬ 
tions  and  hazards  of  the  Cost-Effectiveness  approach  to  clarifying 
choices  in  the  military  area  in  general,  then  develop  some  aspects 
of  the  management  problem  raised  by  these  limitations  and  hazards, 
and  finally  review  these  problems  with  special  reference  to  R  &  D 
decisions . 


I 

To  begin  with,  I  regard  the  CE  technique  as  a  most  valuable  tool 
for  elucidating  choices  on  military  capabilities.  However,  the  mea¬ 
sure  of  its  usefulness  depends  crucially  on  the  sense  of  restraint 
with  which  the  tool  is  applied  and  its  product  appreciated.  From 
this  point  of  view,  I  am  concerned,  not  with  the  vulgar  objections 
to  the  CE  technique,  but  only  with  those  that  its  advocates  themselves 
admit.  Since  these  limitations  are  well-known,  I  will  state  them  in 
highly  summary  fashion. 

First,  as  everybody  knows,  the  CE  approach  has  unlimited  power 
when  normative  problems  are  answered  by  givens  so  that  we  face  a  pure 
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and  simple  problem  of  maximization,  and  when  all  costs  as  well  as 
benefits  are  known  and  can  be  quantified.  This  means  that  the  tech¬ 
nique  is  at  its  most  useful  when  the  objective  or  output  is  definitely 
fixed,  that  is.  when  there  is  only  one  dependent  variable,  and  the 
sole  task  is  to  minimize  the  costs  which  are  readily  and  accurately 
measured.  In  these  cases,  alternative  means  to  achieving  the  objec¬ 
tive  differ  only  in  this  key  variable,  and  choosing  the  cheapest 
means  in  this  sense  is  the  only  problem. 

On  the  other  hand,  it  is  generally  agreed  that  the  CE  approach 
is  less  useful  in  clarifying  choices  when  the  employment  of  different 
means  leads  to  appreciably  different  outputs.  Its  usefulness  is  the 
more  restricted,  the  more  incommensurable  the  outputs  and  the  more 
appreciable  and  unmeasurable  the  social  costs  other  than  those  quan¬ 
tifiable  in  terms  of  money;  that  is,  the  usefulness  of  the  technique 
is  the  more  limited,  the  less  the  problem  is  capable  of  uniform 
quantification.  This  limitation  is  really  obvious  since  rational 
decision-making  requires  us  to  maximize  the  value  of  all  benefits 
minus  the  value  of  all  costs  or  disadvantages.  I  stress  that  the 
utility  of  the  technique  depends  very  importantly  on  the  completeness 
with  which  costs  and  benefits  are  analyzed.  1  stress  particularly 
other  costs  than  money  costs;  for  these  can  be  of  great  variety  and, 
it  seems  to  me,  they  are  easily  lost  sight  of.  They  may  be  political, 
as  when  a  particular  choice  causes  great  inconvenience  to  an  ally;  or 
when  choice  engenders  a  sharp  decline  in  the  morale  of  a  military 
service.  Surely,  service  morale  is  an  asset  and  its  deflation  is, 
as  such,  a  disadvantage.  We  might  want  to  accept  this  disadvantage 
if  the  net  benefits  of  a  choice  promise  to  be  very  substantial,  but 
we  should  hardly  ignore  or  neglect  it.  To  the  extent  that  costs  and 
benefits  cannot  be  measured  with  accuracy,  and  to  the  extent  that 
the  problem  is  one  of  deciding,  in  an  inherently  subjective  manner, 
between  different  sets  of  costs  and  benefits,  problems  of  choice 
are  insusceptible  to  rigorous  economic  analysis.* 

*Cf.  James  R.  Schlesinger,  ’’Quantitative  Analysis  and  National 
Security,”  World  Politics,  Vol.  15,  p.  305. 
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A  second  great  limitatton  of  the  CE  approach  results  from 
imperfect  information.  In  the  military  area,  various  incalculable 
uncertainties  must  be  faced  often.  Costs  may  be  uncertain,  technol¬ 
ogy  may  be  uncertain,  the  properties  of  military-conflict  situations 
may  be  uncertain,  and  the  reactions  and  capabilities  of  potential 
enemy  nations  are  apt  to  be  uncertain.  This  last  uncertainty  is  of 
particular  import;  it  is  imperative  that  military  choices  be  examined 
within  a  framework  of  interaction.  An  opponent's  response  to  our 
choices  may,  after  all,  curtail  or  altogether  nullify  the  advantages 

we  seek.  Nor  is  it  enough  to  recognize  the  conflict  aspects  of  this 

1 

problem  of  interaction.  The  possibilities  of  tacit  or  formal  co¬ 
operation,  that  is,  of  non-zero-8um  types  of  situations,  may  be 
equally  significant. 

I  would,  somewhat  recklessly,  add  a  third  potential  limitation 
of  the  CE  approach,  and  that  is  the  salience  it  inevitably  attributes 
to  the  criterion  of  purely  monetary  costs.  Of  course,  I  realize 
that,  in  a  world  of  scarce  resources,  this  must  be  an  important  yard¬ 
stick.  But,  making  the  outrageous  assumption  for  the  moment,  that 
other  costs  or  benefits  lend  themselves  as  readily  to  quantification 
as  estimated  money  costs  do,  should  we  not  consider  whether,  in 
societies  becoming  ever  more  affluent,  monetary  costs,  though  impor¬ 
tant,  should  not  be  expected  to  decline  in  importance  relatively  to 
other  values,  positive  or  negative.  For  example,  would  the  minimiza¬ 
tion  of  military  conflict,  or  of  loss  of  human  life  in  the  event  of 
conflict,  not  gain  importance  in  relation  to  monetary  sacrifice? 

This  consideration  should  certainly  make  a  claim  on  our  attention 
whenever  the  monetary  sacrifice  involved  in  choices  is,  absolutely 
speaking,  rather  small.  Thus,  if  we  wanted  to  apply  the  CE  technique 
to  assessing  the  worth  of  the  cost-effectiveness  approach  itself, 
its  financial  costs  would  probably  be  regarded  as  relatively  trivial, 
and  we  would  look  toward  more  significant  criteria  of  the  disadvantages 

^This  exclusiveness  is  suggested  in  E.  S.  Quade,  "Methods  and 
Procedures,"  in  E.  S.  Quade  (ed.),  Analysis  for  Military  Decisions, 

The  RAND  Corporation,  R-387-PR,  November  1964,  p.  152. 
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of  cost-effectivcness  studies.  Now,  other  criteria  do  not,  un¬ 
happily,  permit  quantification  nearly  as  much  as  money  costs  do. 

But,  to  the  extent  this  is  the  case,  and  accounts  for  our  preference 
to  focus  on  money  costs,  are  we  then  not  saying  that  the  value  of 
this  focus  on  u’onetary  costs  is  derived  from  the  convenient  fact 
that  they  are  capable  of  easy  measurement? 

There  is  general  agreement  that  the  two  major  limitations  I 
stated  first  greatly  restrict  the  usefulness  of  the  CE  approach  in 
making  high-level  decisions  on  military  matters,  for  such  decisions 
do  Involve  choice-of-objective  problems  and  bristle  with  intangibles 
and  uncertainties.  But  it  is  to  complex  problems  of  this  kind  that 
the  CE  technique  is  in  fact  applied,  and  should  be  applied.  To  do 
so  is,  if  rightly  done,  entirely  proper  and  unquestionably  useful. 
Not  only  are  differences  in  money  costa  usually  important,  to  pro¬ 
ceed  rationally  we  must  obviously  also  regard  all  advantages  of  a 
policy  as  a  return,  and  all  disadvantages  as  a  cost,  and  define  the 
best  policy  as  the  one  which  offers  the  largest  margin  of  return 
over  costs. 


II 

However,  it  is  precisely  at  this  point  that  we  encounter  the 
problems  of  management .  One  problem  is  that  the  value  of  CE  analysis 
is  sensitive  to  the  resources  and  time  given  to  it,  and  that  hasty 
analyses  may  do  more  harm  than  good.  A  critical  factor  along  this 
line  may  be  the  tendency  for  the  insistence  of  the  budgetary  cycle 
to  deny  sufficient  time. 

A  second  problem  is  that,  partly  resulting  from  time  pressure, 

CE  studies  are  fragmentary,  that  ail  costs  and  benefits  do  not  receive 
due  attention,  and  that  money  costs  claim  undue  emphasis.  To  put  it 
more  crudely,  we  may  have  too  many  cost  studies  and  not  enough  coat- 
benefit  studies;  analyses  overemphasizing  money  costs  may  just 
pretend  to  be  CE  studies. 

This  leads  me  to  the  related  third  problem  that  the  CE  approach 
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b«  given  no  more  influence  on  decisions  than,  in  view  of  the  inevi¬ 
table  or  practical  limitations  of  CE  studies,  it  can  legitimately 
claim.  The  CE  technique  ray  be  a  scientific  technique;  yet  its  ap¬ 
plication  is  an  art;  that  is  to  say,  an  activity  heavily  dependent 
on  imagination  and  judgment.  There  ia  nothing  wrong  with  CE  as  a 
tool;  but  a  great  deal  could  be  wrong  with  its  exploitation  if  it  ia 
not  governed  by  an  inventive  Imagination,  good  judgment,  and  a  mea¬ 
sure  of  humility.  Let  me  spell  out  briefly  some  additional  dangers 
arising  in  its  exploitation;  end  it  is  clear  to  me  that  nearly  all 
these  dangers  have  been  discovered  and  acknowledged  by  proponents 
of  the  technique. 

The  central  problem  is  that  CE  studies  must  count  for  no  more, 
and  no  less,  than  their  due.  They  got  less  than  their  due  before 
Mr.  McNamara  became  Secretary  of  Defense.  The  question  is  whether 
there  is  not  now  a  tendency  in  the  Department  of  Defense  for  CE 
studies,  suffering  from  the  lack  of  balance  I  have  discussed,  to 
receive  more  than  their  due.  If  it  were  true,  this  would  be  a 
serious  matter,  and  especially  so  in  the  case  of  important,  very 
high-level  decisions  regarding  which,  it  is  generally  agreed,  the 
CE  technique  can  make  only  a  very  limited  contribution,  even  though 
a  valuable  one,  within  these  narrow  limits.  I  do  not  know  that  there 
is  now  a  tendency  in  the  DOD  for  CE  studies  to  be  accorded  excessive 
attention  and  weight.  I  suspect  that  this  is  sc.  I  suspect  rather 
than  know  because  what  evidence  I  have  is  very  little  and  rather  soft. 

The  evidence  is  of  two  kinds.  First,  a  considerable  number  of 
people  who  have  been  close  to  the  decision-making  process,  and  whose 
judgment  I  respect,  have  told  me  that  the  tendency  prevails.  Second, 

I  read  with  cere  Mr.  McNamara's  1963  testimony  on  why  he  preferred  a 
conventionally- fueled  to  a  nuclear-fueled  aircraft  carrier.  In  hit 
very  lengthy  teatimony,  Mr.  McNamara  came  back  again  and  again  to 
the  difference  In  money  costa  but,  though  several  Senators  pressed 
him  with  Intelligent  and  pertinent  question*,  he  never  explained  why 
the  advantagee  of  the  nuclear  carrier  were  not  worth  the  difference 
in  these  coats.  He  contented  himself  with  stating  flatly  that  he 
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did  net  think  they  were,  while  citing  eagerly  and  at  length  some 
dubious  analogies  of  why  he  personally  was  better  off  buying  a 
medium-  rather  than  high-price  automobile,  or  why  a  farmer,  having 
to  transport  produce  to  the  market  from  time  to  time,  might  be  better 
off  with  a  cheaper  and  slower  than  with  a  speedier  and  more  expensive 
truck.  The  trouble  with  these  analogies  is  that  they  explain  the  CE 
principle  —  especially,  a  stripped-down  economy  version  of  it  --  but 
that  they  do  not  explain  at  all  the  superiority  as  a  buy  of  a  conven¬ 
tional  over  a  nuclear  carrier.  Obviously,  Mr.  McNamara  has  far  better 
information  on  the  factors  affecting  his  choice  of  automobile  than  he 
could  possibly  have  about  the  future  utility  of  different  aircraft 
carriers  in  different  contingencies  whose  probabilities  are  unpre¬ 
dictable.  He  also  could  easily  make  some  simple  assumption  about 
the  hypothetical  farmer's  transportation  problem,  while  similar  as¬ 
sumptions  about  the  future  missions  of  aircraft  carriers  are  more 
difficult  to  make.  I  could  not  help  getting  the  strong  impression 
that  the  money-cost  difference  and  the  stripped-down  CE  model  were 
foremost  in  the  Secretary's  mind,  and  that  the  very  complicated 
guesswork  on  possible  demands  on  aircraft  carriers  some  years  hence 
was  not. 

This  leads  me  to  the  proposition  that  the  structure  of  attention 
paid  by  decision-makers  to  complex  problems  of  choice  should  accord 
no  more  salience  to  CE  studies  than  they  can  properly  claim.  This 
precept  is  more  easily  expressed  than  implemented .  Implementation 
requires  that  the  total  problem  be  subjected  to  orderly  conceptualiza¬ 
tion,  that  the  intangibles  as  well  as  the  quantitative  factors  be 
properly  analyzed,  and  that  an  attribution  of  relative  priorities 
guide  the  decision-maker  in  how  to  bring  various  parts  of  the  analysis 
to  bear  on  the  problem  of  choice.  This  last  condition  is  very  impor¬ 
tant  in  a  bureaucracy  in  which  decisions  are  prepared  at  various 
layers  on  a  decentralized  basis.  Even  if  these  rules  are  observed, 
the  danger  remains  that  the  top-level  decision-maker  may  be  exces¬ 
sively  attracted  by  the  neatness  of  quantitative  analysis  and  con¬ 
clusions,  and  that  he  may  neglect  those  parts  of  the  analysis  that 
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are  iffy,  perhaps  obscure,  messy,  and  certainly  hard  to  evaluate. 

Even  a  practitioner  like  Hr.  Hitch  concedes  this  "potential  hazard" 
to  which  CE  studies  may  give  rise.^  One  antidote  is  no  doubt  the 
development  of  effective  models  for  qualitative  analysis.  But  they 
will  be  very  useful  only  if  the  top  decision-maker  does  not  nurse  a 
vulgar  skepticism  regarding  non-economic  models,  and  if  there  are 
personnel  with  an  adequate  range  of  skills  to  design  and  work  these 
models . 

Indeed,  excessive  reliance  on  economists  and  other  experts  on 
quantitative  analysis  is  another  condition  that  may  cause  the  pre¬ 
sentation  of  problems  of  choice  to  be  slanted  in  favor  of  CE  analysis 
that  overemphasizes  money  costs.  It  is  not  only  that  the  CE  experts 
do  not  necessarily  command  expertise  on  essentially  military,  political, 
and  psychological  problems,  it  is  also  that  they  tend  to  have  acquired 
perceptual  propensities  more  suited  to  some  problems  than  to  others. 

In  this  respect,  it  is  interesting  to  note  the  praise  lavished  on 

these  propensities  by  Charles  Hitch.  He  approves  of  "...economic 

2 

choice  as  a  way  of  looking  at  problems...;"  he  lauds  the  "quantified 

3 

common  sense"  of  the  systems  analyst;  and  he  remarks  that  systems 

analysis  "...provides  the  checks  and  balances  so  essential  to  minimiz- 

4 

ing  parochial  viewpoints..."  Hitch  is  quite  right  in  extolling  these 
virtues.  Yet  this  style  of  thought  is  virtuous  only  within  the  proper 
context  and,  when  pressed  beyond,  virtue  could  turn  into  vice. 
"Quantified  common  sense"  may  give  short  shrift  to  the  analysis  of 
Intangibles,  and  systems  analysts  could  develop  a  parochialism  of 
their  own  unless  there  are  checks  and  balances  --  and  I  do  not  mean 
service  biases  --  to  supplement  their  intellectual  habits.  Such 


Cf.  Quade  (ed.),  Analysis  for  Military  Decisions,  pp.  326-327; 
see  also  Quade,  "Methods  and  Procedures,"  ibid.,  p.  170;  Schlesinger, 
op.  cit . ,  p .  300 . 

2 

Charles  J.  Hitch,  Decision-Making  for  Defense,  Berkeley,  1965, 

p.  53 „ 


3Ibid.,  p.  55. 
4Ibld.,  p.  57. 
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checks  and  balances  demand  sophisticated  personnel  with  a  different 
range  of  expertise. 

The  proponents  of  CE  studies  admit  that  this  management  problem 
exists,  although  they  express  it  perhaps  more  vaguely  than  I  have 
done.  Hr.  Hitch  avers  that  -the  dangers  are  known  and  hence  controlled. 
But  Schlesinger  has  abserved  that  "...a  ritualistic  recitation  of  the 
dangers  of  excessive  quantification  characteristically  precedes  the 
attempt  to  push  quantitative  analysis  too  far."^  Surely,  the  mere 
argument  by  the  practitioners  that  the  problem  is  under  control  will 
not  do.  The  argument  may  be  truthful  or  not.  One  must,  therefore, 
insist  that  whether  or  not  the  dangers  of  CE  have  been  avoided  and 
are  avoided  in  the  DOD  is  an  empirical  question  that  cannot  be 
settled  by  argument.  To  convince  me  that  they  are  avoided  would  re¬ 
quire  a  thorough  study  of  past  problem-solving  by  impartial  researchers. 
This  could  be  done  by  a  number  of  properly  selected  and  properly 
conducted  postmortems.  I  do  not  in  fact  know  whether  or  not  such 
postmortems  have  been  undertaken.  If  they  have  not,  the  determina¬ 
tion  with  which  the  CE  technique  is  applied  to  problems  of  great 
national  importance  makes  it  imperative  that  such  a  study  be  under¬ 
taken.  To  do  so  is  in  fact  in  the  spirit  of  systems  analysis. 

There  is  another  way  of  studying  this  management  problem  empir¬ 
ically.  Psychologists  could  explore  the  relevant  conditions  that 
tend  to  slant  the  perception  and  prejudice  the  attention  of  policy¬ 
makers  and  their  assistants  when  confronted  with  highly  complex 
problems  of  choice.  I  do  not  mean  that  decision-makers  should  be 
psychoanalyzed.  What  I  have  in  mind  is  that  experimental  psychol¬ 
ogists  have  already  elucidated  many  of  the  conditions  that  affect 
perception  and  attention,  and  that  it  should  be  possible  for  them 
to  explore  these  problems  within  the  context  of  our  interest.  How 
does  the  presentation  of  policy  analysis,  some  quantitative  and  some 
qualitative,  some  based  on  solid  information,  and  some  on  surmise, 
impinge  on  perception  and  attention?  If  such  empirical  efforts 
produced  more  knowledge  than  we  now  have,  we  would  be  in  a  better 
position  to  engineer  improvements  in  the  art  of  exploiting  CE 

1 Schlesinger ,  op.  cit.,  p.  306. 
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analyais.  If  this  advocacy  on  fact-finding  research  strikes  anyone 
as  being  superfluous,  let  me  remind  him  of  one  thing  we  do  know 
about  human  behavior,  namely,  that  cognizance  of  a  problem  does  not 
lead  automatically  to  its  solution. 

Ill 

Now  I  will  turn  briefly  to  a  review  of  our  management  problem 
with  particular  reference  to  choices  in  military  R  &  D.  The  limita¬ 
tions  on  the  usefulness  of  CE  studies,  and  the  management  risks  to 
which  such  studies  give  rise,  are  much  greater  when  the  CE  technique 
is  applied  to  R  &  D  choices  than  to  military  choices  in  general. 
Clearly,  to  decide  on  preferring  a  conventional  aircraft  carrier  to 
a  nuclear-powered  carrier  in  1963  was  far  easier  a  problem  (though 
not  apparently  an  easy  problem)  than  to  decide  earlier  on  that  a 
nuclear  carrier  should  be  developed. 

In  the  case  of  R  &  D  choices,  the  uncertainties  tend  to  be 
greater.  As  experience  shows  abundantly,  financial  costs  are  hard 
to  estimate;  technological  advance  is  difficult  to  predict,  and  the 
benefits  are  hard  to  evaluate.  After  all,  R  &  D  outputs  will  affect 
military  capabilities  only  with  a  considerable  time  lag  and  --  during 
this  time  --  the  relevant  military,  technological,  and  political 
environment  may  undergo  substantial  changes  that  impinge  on  the  value 
of  a  weapon  system,  or  of  a  strategy  for  which  it  is  designed.  Above 
all,  the  capabilities  of  potential  enemies  may  change  significantly, 
in  part  perhaps  as  a  result  of  their  reaction  to  our  R  &  D  choices. 
Even  the  actual  use  of  evolving  weapons  is  hard  to  predict;  at  least, 
history  provides  numerous  examples  of  new  weapons  finding  uses  quite 
different  from  what  they  were  originally  intended  for. 

The  impingement  of  these  conditions  that  are  hard  to  cope  with 
will,  of  course,  vary  a  great  deal.  Their  impingement  will  tend  to 
be  the  greater,  the  more  R  &  D  projects  involve  true  innovation 
rather  than  marginal  improvement.  This  proposition  follows  from  the 
fact  that  the  more  innovation  is  involved,  the  greater  will  be  the 
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uncertaintiea .  Their  impingement  will  also  tend  to  be  the  greater, 
the  earlier  the  phase  of  R  &  D,  again  because  at  this  stage  the 
uncertainties  are  greater. 

It  i 8  true  that  the  CE  technique  is  not  applied  at  present  to 
the  most  initial  phases  of  R  &  D  proposals  --  Research  and  Exploratory 
Development  --  as  long  as  the  estimated  costs  fall  short  of  a  certain 
amount.  The  questions  are  whether  the  cut-off  is  well-chosen  and 
whether  a  fixed  cut-off  is  in  principle  sound. 

At  any  rate,  if  the  enthusiastic  practitioner  of  CE  is  apt  to 
exaggerate  the  assistance  his  technique  can  give  to  the  policy-maker, 
this  is  especially  dangerous  in  the  case  of  R  &  D  choices.  The 
worst  danger  would  be  if  the  insusceptibility  to  CE  studies  of  highly 
innovative  projects  led  to  a  bias  favoring  projects  that  are  more 
susceptible.  In  the  case  of  R  &  D  proposals  involving  a  high  degree 
of  innovation,  it  would  seem  relatively  more  important  to  seek  advice 
from  people  adept  at  making  conjectures  about  the  future  military  and 
political  environment,  and  on  the  military  needs  which  changes  in 
this  environment  may  generate.  Only  by  doing  so  can  we  reduce 
consumer  ignorance  at  the  higher  levels  of  decision-making  where 
the  insertion  of  judgment  is  crucial. 

E j  way  of  conclusion  I  will  repeat  my  plea  for  empirical  studies 
of  the  values  and  dangers  of  CE  guidance.  Surely,  to  find  out  more 
about  how  to  manage  CE  is  specially  important  with  reference  to 
military  R  &  D.  Indeed,  the  case  for  CE  would  gain  in  strength  if 
we  learned  more  about  the  art  of  its  application.  If  we  did  so, 
and  particularly  if  we  insisted  on  comprehensive  CE  studies,  I 
would  certainly  conclude:  let  us  have  more  of  them. 


